Semi-Markov processes and α-invariant distributions

The KTH Visit in Semi-Markov Processes

Solving Generalized Semi-Markov Processes using Continuous Phase-Type Distributions

We introduce the generalized semi-Markov decision process (GSMDP) as an extension of continuous-time MDPs and semi-Markov decision processes (SMDPs) for modeling stochastic decision processes with asynchronous events and actions. Using phase-type distributions and uniformization, we show how an arbitrary GSMDP can be approximated by a discrete-time MDP, which can then be solved using existing M...

Exact Distributions for Reward Functions on Semi-markov and Markov Additive Processes

The distribution theory for reward functions on semi-Markov processes has been of interest since the early 1960s. The relevant asymptotic distribution theory has been satisfactorily developed. On the other hand, it has been noticed that it is difficult to find exact distribution results which lead to the effective computation of such distributions. Note that there is no satisfactory exact distr...

q-distributions and Markov processes

We consider a sequence of integer{valued random variables Xn; n 1; representing a special Markov process with transition probability the transition probability is given by n;` = q n+`+ and n;` = 1 ? q n+`+ , we can nd closed forms for the distribution and the moments of the corresponding random variables, showing that they involve functions such as the q{binomial coeecients and the q{Stirling n...

Semi-markov Decision Processes

Considered are infinite horizon semi-Markov decision processes (SMDPs) with finite state and action spaces. Total expected discounted reward and long-run average expected reward optimality criteria are reviewed. Solution methodology for each criterion is given, constraints and variance sensitivity are also discussed.